[Backend Tester] Write report progressively #13308
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13308
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
bot
added
the
CLA Signed
| write_csv_header(f) | ||
| _active_session.has_written_report_header = True | ||
|
|
||
| write_csv_row(summary, f) |
There was a problem hiding this comment.
This implies we crash when we run into some failure? Can we try to catch and fail gracefully? Instead of assuming that we can crash anytime.
There was a problem hiding this comment.
This is mainly for memory corruption, which happens in a few cases. It might be possible to catch the SIGSEGV or other native fault, but I don't know what state the process is it, so I'm not sure if it's recoverable. Open to suggestions.
| @@ -262,6 +300,15 @@ def log_test_summary(summary: TestCaseSummary): | |||
| if _active_session is not None: | |||
| _active_session.test_case_summaries.append(summary) | |||
|
|
|||
| if _active_session.report_path is not None: | |||
There was a problem hiding this comment.
can multiple subprocesses write to this simultaneously?
There was a problem hiding this comment.
Probably not safely. What is the use case? Are you thinking we parallelize tests between processes? That seems nice to have, though I'd be inclined to deal with concurrency issues if/when we add that feature.
Append to the report file line by line after each test, rather than all at the end. This ensures that report data is available if the test run is aborted or hit with an unrecoverable native crash.